Skip to content

Fix external type resolution failing when the type's dependencies are not deployed with the generator - #11465

Open
rhurey wants to merge 3 commits into
mainfrom
rhurey/assembly_closure
Open

Fix external type resolution failing when the type's dependencies are not deployed with the generator#11465
rhurey wants to merge 3 commits into
mainfrom
rhurey/assembly_closure

Conversation

@rhurey

@rhurey rhurey commented Jul 30, 2026

Copy link
Copy Markdown
Member

Problem

 @@alternateType(..., { identity, package, minVersion }, "csharp")  silently had no effect whenever the referenced type's base type (or any type in its signature) lived in a package the generator does not itself deploy. Instead of referencing the external type, the generator emitted its own copy of the model.

Reproducing with  Azure.AI.Projects.Agents , 12 tool models mapped to  Azure.AI.Extensions.OpenAI  were all still generated, each reporting:

unsupported-external-type: External type 'Azure.AI.Extensions.OpenAI.BingGroundingTool' could not be resolved:
package 'Azure.AI.Extensions.OpenAI' (>= 3.0.0-alpha.20260729.1) was not found in the NuGet cache or any configured feed.

The diagnostic was misleading — the package was present and was found.  ExternalTypeReferenceResolver  loads the package assembly with  Assembly.Load(byte[])  into the default  AssemblyLoadContext , which resolves dependencies only from the generator's own trusted-platform-assemblies list.  BingGroundingTool  derives from  OpenAI.Responses.ResponseTool , and  OpenAI.dll  is not deployed with the generator, so  Assembly.GetType(identity, throwOnError: false)  returned  null  without throwing. That null was interpreted as "type not found", and generation proceeded as if no external mapping existed.

Fix

Adds  NugetAssemblyResolver , which satisfies the transitive references of external-type assemblies from the NuGet global packages folder via an  AssemblyLoadContext.Default.Resolving  hook. Two details drive the design:

Dependencies are pinned from the package dependency closure, not from assembly versions. An assembly reference only carries an assembly version, which for many packages is deliberately lower than the package version that ships it —  System.ClientModel  1.14.0 ships assembly version  1.9.0.0 . Resolving that as a package version selects the unrelated 1.9.0 package, and the resulting type-load failure surfaces as  MissingMethodException: Method not found: 'System.BinaryData IPersistableModel.Write(ModelReaderWriterOptions)' . The resolver instead walks the package's  .nuspec  transitively, picking the framework-nearest dependency group and unifying duplicates by highest version, mirroring what NuGet would select.

An assembly the host already provides is preferred over the cache, regardless of version. The default context only asks the hook to resolve a name it could not satisfy itself, but it will still satisfy other, lower-versioned requests for that same name from its own deployment. Loading a second copy from the cache therefore leaves two assemblies with the same simple name in the process, and any type they both declare exists twice. Concretely:  Azure.AI.Extensions.OpenAI  requests  System.Memory.Data  10.0.0.9 while the generator deploys 10.0.0.3; loading 10.0.9 from the cache produces two  System.BinaryData  types, and every signature mentioning it fails to bind. Reusing the host's copy keeps exactly one identity per simple name.

Resolved dependencies are also registered as Roslyn metadata references so generated code that references them still binds in the generated-code workspace.

Two supporting changes:

•  NugetPackageResolver.FindPackageAssemblyInVersion  looks up one exact installed version and selects the  lib/  asset closest to the framework the generator is running on. The existing  FindPackageAssembly  probes  PreferredDotNetFrameworkVersions  and so prefers  netstandard2.0 , which is correct for collecting compile-time references but not for assemblies loaded into a live .NET process, where a  netstandard2.0  asset can bind against compatibility shims that duplicate shared-framework types.
•  ExternalTypeReferenceResolver  now records a specific failure reason per resolution path, and  TypeFactory  reports it in  unsupported-external-type . The previous message claimed the package was missing regardless of the actual cause; failures now distinguish "package not found", "assembly could not be read/loaded", and "assembly was loaded but does not declare the type, or one of the type's dependencies could not be resolved".

Validation

• New  TryResolve_ResolvesTypeWhoseBaseTypeLivesInAnotherPackage  builds two interdependent fake NuGet packages and asserts the derived type resolves with a readable  BaseType . Confirmed to fail without the fix, with exactly the symptom above.
• New  TryResolve_ReportsFailureReasonWhenTypeMissingFromAssembly , plus a  GetFailureReason  assertion on the existing unknown-package test.
•  Microsoft.TypeSpec.Generator.Tests  1783/1783;  Microsoft.TypeSpec.Generator.ClientModel.Tests  1536/1536.
• End-to-end against  Azure.AI.Projects.Agents : all 12  unsupported-external-type  warnings are gone and none of the 12 tool models are generated.

Notes for reviewers

•  FakeNuGetPackage.Create  gained an optional  referencedAssemblyPaths  parameter so tests can build packages that reference each other.
•  ExternalTypeReferenceResolver.Reset()  now tolerates  InvalidOperationException . It dereferences  CodeModelGenerator.Instance , which is not available in test  SetUp  before a mock generator is loaded — pre-existing fragility, surfaced by the new tests.
• The resolver keys its cache on assembly simple name and returns cached failures, so a missing dependency is probed once per process.

@microsoft-github-policy-service microsoft-github-policy-service Bot added the emitter:client:csharp Issue for the C# client emitter: @typespec/http-client-csharp label Jul 30, 2026
@pkg-pr-new

pkg-pr-new Bot commented Jul 30, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-csharp@11465

commit: 0b760e9

@github-actions

Copy link
Copy Markdown
Contributor

No changes needing a change description found.

// same simple name in the process, and any type they both declare (System.Memory.Data's
// System.BinaryData, for example) would exist twice, so signatures mentioning it fail to bind
// with a MissingMethodException. Reusing the host's copy keeps exactly one identity per name.
var assembly = UseHostAssembly(simpleName) ?? Probe(globalPackagesFolder, simpleName, name.Version);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unconditionally prefers the host assembly by simple name, even when the package closure selected a newer/incompatible version. That can recreate the same MissingMethodException class of failure this change is intended to fix—for example, the PR notes that Extensions requests System.Memory.Data 10.0.0.9 while the host carries 10.0.0.3. If maintaining a single type identity requires using the host copy, please verify compatibility with the closure requirement and report a clear resolution failure when it is not compatible rather than silently substituting it.

--generated by Copilot

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed, though not by failing. Made the substitution visible instead.

UseHostAssembly now compares the host's version against the requested one and records a note when the host's copy is older. DescribeDowngradedDependencies() is appended to the "could not be loaded" and "does not declare the type" failure reasons, so a type that fails because of the substitution now says so rather than looking like a plain missing type.

I deliberately did not make the mismatch fatal, because there is no version-compatible answer available when the host and the closure disagree:

  • Load the closure's copy: two assemblies with the same simple name end up in the process. System.Memory.Data declares System.BinaryData rather than forwarding it, so BinaryData then exists twice and every signature mentioning it fails to bind. That is exactly the MissingMethodException this PR was written to fix.
  • Return null: the load fails outright with Could not load file or assembly 'System.Memory.Data, Version=10.0.0.9'. I tried this variant; it regresses the working scenario.
  • Use the host's copy: one identity per simple name, and in practice it works, because assembly versions are routinely older than the reference that names them.

The concrete case here is the generator deploying System.Memory.Data 10.0.0.3 while Azure.AI.Extensions.OpenAI asks for 10.0.0.9. Both are BinaryData-over-corlib shims; the request is only unsatisfiable because the .NET SDK normally prunes this package on net10.0 and the generator carries it explicitly.

Note also that the hook only ever runs for names the default context could not satisfy on its own. It cannot prevent that same context from satisfying other, lower-versioned requests for the same name from its own TPA list, so declining here does not buy isolation - it just loses the load.

}

[Test]
public void TryResolve_ResolvesTypeWhoseBaseTypeLivesInAnotherPackage()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This regression test does not exercise RegisterPackageClosure/WalkPackage: FakeNuGetPackage.Create writes only the DLL under lib/netstandard2.0 and does not create a .nuspec, so closure walking immediately takes the missing-nuspec fallback. Because both fake package and assembly versions are also 1.0.0, the test passes through the assembly-version fallback. Please add a dependency group to a fake .nuspec and deliberately make the dependency package version differ from its assembly version (for example package 2.0.0 containing assembly 1.0.0.0) so the new closure/version-selection path is actually required for the test to pass.

--generated by Copilot

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify the requested scope: please add only the hermetic unit test described above. We can leave validation against real published NuGet packages as a manual E2E check for now.

--generated by Copilot

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and that is how it was done. The test builds its packages on disk under a temp NUGET_PACKAGES folder with Roslyn and never contacts a feed. The real Azure.AI.Extensions.OpenAI / OpenAI / System.ClientModel graph stays a manual end-to-end check.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, and correct. FakeNuGetPackage.Create wrote only lib/netstandard2.0/{pkg}.dll and no .nuspec, so WalkPackage hit its missing-nuspec early return, _closureVersions stayed empty, and the test passed entirely through the assembly-version fallback. It was not testing the code it was written for.

Fixed by emitting a real .nuspec with a .NETStandard2.0 dependency group, and giving the test helper an assemblyVersion parameter separate from the package version. The test now builds:

  • Test.Dependency.Base package 2.0.0 / assembly 1.0.0.0, declaring DependencyBaseType
  • Test.Dependency.Base package 3.0.0 / assembly 1.0.0.0, a decoy without that type
  • Test.Dependent.Leaf 1.0.0, whose .nuspec depends on Test.Dependency.Base [2.0.0, )

The leaf's assembly reference carries version 1.0.0.0, so the fallback searches for the highest package at or above 1.0.0 and lands on the 3.0.0 decoy. Only the .nuspec closure identifies 2.0.0.

Verified by neutering RegisterPackageClosure and re-running:

Failed TryResolve_ResolvesTypeWhoseBaseTypeLivesInAnotherPackage
  Could not load type 'Test.Dependency.Base.DependencyBaseType' from assembly
  'Test.Dependency.Base, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'

Closure walking is now strictly required for the test to pass.

/// type whose base type lives in a dependency. The external type then appears unresolvable and is
/// generated instead of referenced.
/// </remarks>
internal static class NugetAssemblyResolver

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping assembly resolution as a separate type makes sense, but can this be a per-generator instance owned by ExternalTypeReferenceResolver's cache state rather than process-wide static state? The closure versions, resolved assemblies, in-progress set, load-context hook, and debug callback all belong to one generation run. Instance ownership would prevent state/type-binding decisions from leaking between sequential generator invocations and would make teardown and isolated testing explicit instead of relying on every static collection and event subscription being reset correctly.

--generated by Copilot

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. NugetAssemblyResolver is an internal sealed class instantiated lazily by ExternalTypeReferenceResolver's per-generator CacheState, so it shares that state's lifetime and is dropped by Reset(). _resolved, _closureVersions, _walkedPackages and _registeredReferences are all instance fields now, [ThreadStatic] became ThreadLocal<HashSet<string>>, and the debug sink and metadata-reference registration are injected as delegates rather than reaching for CodeModelGenerator.Instance.

Two things stay static, because they are genuinely process-wide rather than per-run:

  • AssemblyLoadContext.Default exposes exactly one Resolving event. A single handler is installed on first use and dispatches to Volatile.Read(ref _active); Activate() / Deactivate() swap the target. Subscribing per instance would be a leak - CacheState lives in a ConditionalWeakTable with no deterministic disposal, so handlers would accumulate across runs in one process, which is the norm under test.
  • Assemblies loaded into the default context can never be unloaded, so Deactivate() only detaches the bookkeeping.

This is safe because FindHostAssembly reads process truth directly from AssemblyLoadContext.Default.Assemblies rather than mirroring it in a static. The instance dictionaries are therefore pure caches, and a fresh instance re-derives the same answers.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a long-standing failure mode in the C# generator’s external-type mapping where @@alternateType(..., "csharp") could silently stop working when the referenced external type depended on assemblies not deployed with the generator. It introduces NuGet-cache-based dependency probing for assemblies loaded into the default AssemblyLoadContext, and improves diagnostics so failures report the real reason (missing package vs. load/type resolution problems).

Changes:

  • Add NugetAssemblyResolver to resolve transitive external-assembly dependencies from the NuGet global packages folder via AssemblyLoadContext.Default.Resolving, and register resolved dependencies as Roslyn metadata references.
  • Update NugetPackageResolver to support exact-version assembly lookup and runtime-framework-nearest lib/ selection when loading assemblies into the live generator process.
  • Expand unit tests and fake NuGet package infrastructure to cover dependency-chain resolution and improved failure reason reporting.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/test/Utilities/TestData/ExternalTypeReferenceResolverTests/DependentPackageSource.cs Adds template source for a dependent package whose type inherits from a type in another package.
packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/test/Utilities/TestData/ExternalTypeReferenceResolverTests/DependencyPackageSource.cs Adds template source for the dependency/base package used by new resolver tests.
packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/test/Utilities/ExternalTypeReferenceResolverTests.cs Adds tests for resolving external types with transitive dependencies and for improved failure reporting; enhances fake package creation helper.
packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/test/common/FakeNuGetPackage.cs Allows fake packages to compile against referenced assemblies to model inter-package dependencies.
packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/src/Utilities/NugetPackageResolver.cs Adds exact-version assembly lookup and runtime-framework-nearest lib/ asset selection.
packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/src/Utilities/NugetAssemblyResolver.cs Introduces NuGet cache-based dependency resolution for assemblies loaded into the default AssemblyLoadContext.
packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/src/Utilities/ExternalTypeReferenceResolver.cs Integrates the resolver hook, records per-attempt failure reasons, and resets resolver state more robustly.
packages/http-client-csharp/generator/Microsoft.TypeSpec.Generator/src/TypeFactory.cs Updates unsupported-external-type details to prefer the resolver’s concrete failure reason.

…st require closure walking

Make NugetAssemblyResolver an instance owned by ExternalTypeReferenceResolver's
per-generator cache state instead of a process-wide static. Only the default
AssemblyLoadContext's single Resolving event stays static; it dispatches to the
currently active resolver so repeated runs in one process cannot accumulate
handlers.

Record when a dependency is satisfied by an older copy the generator deploys and
surface those notes in the resolution failure reason, so a type that fails to
load because of the substitution reports why. Preferring the host copy remains
the behavior: loading the closure copy would leave two assemblies with the same
simple name in the process, which is the condition the fix exists to avoid.

Emit a .nuspec from FakeNuGetPackage and give the dependency test a package
version that differs from its assembly version, plus a higher-versioned decoy
package that lacks the base type. The assembly-version fallback selects the
decoy, so the test now passes only when the dependency is pinned from the
.nuspec closure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a3c5d2db-ec93-428f-b474-05e7e58cad60
Comment on lines +131 to +143
var versionDir = Path.Combine(globalPackagesFolder, packageId.ToLowerInvariant(), version.ToLowerInvariant());
string? nuspecPath;
try
{
nuspecPath = Directory.Exists(versionDir)
? Directory.EnumerateFiles(versionDir, "*.nuspec").FirstOrDefault()
: null;
}
catch (Exception ex)
{
Debug($"Failed to enumerate '{versionDir}' while walking package dependencies: {ex.Message}");
return;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we considered re-utilize the existing TryFindPackageInCache method in NugetPackageDownloader ? It calls the nuget SDK to delegate this search to their library vs having to handwrite it ourselves.

}
catch (InvalidOperationException)
{
// No generator is installed (e.g. a unit test resolving types outside of a generation run).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks a bit odd. If this happened in a unit test, should we ensure the generator is initialized as part of the test setup instead?

@@ -0,0 +1,356 @@
// Copyright (c) Microsoft Corporation. All rights reserved.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be great if we can add unit tests for this class and mock some of these methods to ensure things work as expected, and also ensure there are no unexpected crashes if things don't. Assembly loading is hard to get right in my experience 🙂 . Feel free to ignore if you feel the existing tests you added cover failure scenarios.

Match the failure paths and avoid allocating an identical result twice.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a3c5d2db-ec93-428f-b474-05e7e58cad60
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

emitter:client:csharp Issue for the C# client emitter: @typespec/http-client-csharp

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants